Role Assignment for Caching

ABSTRACT

A system, method, and computer program are disclosed to provide an ad-hoc network of network nodes of caching content at the network nodes. A coordination server performing as a manager or controller determines the network node to cache content for a group of user equipments (UEs) attached to a first base station. The coordination server reassigns the network node that cache the content for the group of UEs when a user equipment (UE) moves from the first base station to a second base station.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/298,072, filed Mar. 11, 2019, which claims priority under 35 U.S.C. §119(e) to U.S. Provisional Pat. App. No. 62/641,955, filed Mar. 12,2018, each of which is hereby incorporated by reference in its entiretyfor all purposes. This application also hereby incorporates byreference, for all purposes, each of the following U.S. PatentApplication Publications in their entirety: US20170013513A1;US20170026845A1; US20170055186A1; US20170070436A1; US20170077979A1;US20170019375A1; US20170111482A1; US20170048710A1; US20170127409A1;US20170064621A1; US20170202006A1; US20170238278A1; US20170171828A1;US20170181119A1; US20170273134A1; US20170272330A1; US20170208560A1;US20170288813A1; US20170295510A1; US20170303163A1; and US20170257133A1.This application also hereby incorporates by reference U.S. Pat. No.8,879,416, “Heterogeneous Mesh Network and Multi-RAT Node Used Therein,”filed May 8, 2013; U.S. Pat. No. 9,113,352, “HeterogeneousSelf-Organizing Network for Access and Backhaul,” filed Sep. 12, 2013;U.S. Pat. No. 8,867,418, “Methods of Incorporating an Ad Hoc CellularNetwork Into a Fixed Cellular Network,” filed Feb. 18, 2014; U.S. patentapplication Ser. No. 14/034,915, “Dynamic Multi-Access Wireless NetworkVirtualization,” filed Sep. 24, 2013; U.S. patent application Ser. No.14/289,821, “Method of Connecting Security Gateway to Mesh Network,”filed May 29, 2014; U.S. patent application Ser. No. 14/500,989,“Adjusting Transmit Power Across a Network,” filed Sep. 29, 2014; U.S.patent application Ser. No. 14/506,587, “Multicast and BroadcastServices Over a Mesh Network,” filed Oct. 3, 2014; U.S. patentapplication Ser. No. 14/510,074, “Parameter Optimization and EventPrediction Based on Cell Heuristics,” filed Oct. 8, 2014, U.S. patentapplication Ser. No. 14/642,544, “Federated X2 Gateway,” filed Mar. 9,2015, and U.S. patent application Ser. No. 14/936,267, “Self-Calibratingand Self-Adjusting Network,” filed Nov. 9, 2015; U.S. patent applicationSer. No. 15/607,425, “End-to-End Prioritization for Mobile BaseStation,” filed May 26, 2017; U.S. patent application Ser. No.15/803,737, “Traffic Shaping and End-to-End Prioritization,” filed Nov.27, 2017, each in its entirety for all purposes, having attorney docketnumbers PWS-71700US01, US02, US03, 71710US01, 71721US01, 71729US01,71730US01, 71731US01, 71756US01, 71775US01, 71865US01, and 71866US01,respectively. This document also hereby incorporates by reference U.S.Pat. Nos. 9,107,092, 8,867,418, and 9,232,547 in their entirety. Thisdocument also hereby incorporates by reference U.S. patent applicationSer. No. 14/822,839, U.S. patent application Ser. No. 15/828,427, U.S.Pat. App. Pub. Nos. US20170273134A1, US20170127409A1 in their entirety.This document also hereby incorporates by reference U.S. patentapplication Ser. No. 14/822,839 in its entirety. The purposes for theabove incorporations by reference include at least to provide detailedinformation about the features and functionality of the ParallelWireless Converged Wireless System (CWS or RAN) and HetNet Gateway (HNGor (coordinator) products.

BACKGROUND

In a typical deployment scenario for a UniRAN in the operator network, aUniRAN could be functioning in either one of two possible modes ofoperation, as a gateway mode or as a mesh mode. In a gateway mode ofoperation, the gateway mode has access to an inter-connect networkthough which it establishes connectivity to the cloud. Theinter-connection could either be wired or wireless (e.g. Long TermEvolution or LTE). In a mesh mode, the UniRAN has no direct connectionto the core or infrastructure provider network. Its only means ofconnecting to the network is over mesh-links. Establishment of the meshnetwork is covered in other documents.

A video cache for the UE is generally implemented either as a mastercache in the core network or as a split cache in the core network andradio access network (RAN). A master cache is where unencrypted datafrom the Packet Gateway (PGW) is visible or a Deep Packet Inspection(DPI) technique are used, reduces the latency as compared to thescenario where there is no cache or the cache is maintained at thecontent hosting site/server. However, this mode suffers from latencyrelated problems.

A split cache in the core network and radio access network (RAN) is whenthe master cache as defined above is supported by a slave cache hostedat the serving eNodeB/RAN/AP where the UE is anchored. This techniqueaddresses the latency issues experienced when just one level cache ismaintained at the core. This can also work in case of a network wherethe eNodeB moves with the UE as in some vehicular implementations.However, this mode is limited to a macro eNodeB based network wherehandovers are extremely few as in the case of handover, the entire cacheneeds to be purged. This option does not work for a small cell network.

Currently, there is no way to implement video caching at the edge thatcan handle the flexibility requirements needed by our mesh. For example,video caching at any particular node in the mesh may be limited byinsufficient number of intra-mesh links or insufficient bandwidth. Also,when mesh nodes enter and exit, a need exists to reallocate the meshcaching node.

SUMMARY OF THE INVENTION

The invention relates generally to caching in a wireless network andmore specifically to caching for a UE attached to a base station whenthe UE roams to a new base station.

In a first embodiment, a method for caching content at mesh networknodes includes assigning a first network node as a primary slave cachingnode to cache content for a user equipment (UE) attached to a first basestation. The method further includes determining the UE has moved fromthe first base station to a second base station. The method alsoincludes determining whether the first network node should continue toact as the primary slave caching node for the UE after it has moved tothe second base station and using the first network node as the cachingnode when the determination is that the first network node shouldcontinue to act as the primary slave caching node for the UE. The methodfurther includes designating a second network node as the primarycaching node to cache content for the UE attached to a second basestation when the determination is that the first network node should notcontinue to act as the primary slave caching node for the UE.

In a second embodiment, a non-transitory computer-readable medium isdisclosed, containing instructions which, when executed, causes one ormore processors to perform operations for caching content at meshnetwork nodes. The computer readable medium includes instructions forassigning a first network node as a primary slave caching node to cachecontent for a user equipment (UE) attached to a first base station. Themedium further includes instructions for determining the UE has movedfrom the first base station to a second base station. The medium alsoincludes instructions for determining whether the first network nodeshould continue to act as the primary slave caching node for the UEafter it has moved to the second base station and using the firstnetwork node as the primary slave caching node when the determination isthat the first network node should continue to act as the primary slavecaching node for the UE. The medium further includes instructions fordesignating a second network node as a primary slave caching node tocache content for the UE attached to a second base station when thedetermination is that the first network node should not continue to actas the primary slave caching node for the UE.

In a third embodiment, a system is disclosed for caching content at meshnetwork nodes. The system includes a first and second base station, acoordinating server, one or more primary slave caching nodes, a cachingnode and a UE.

A role-based system is described in which a primary cache and asecondary cache are present in a mesh, though a mesh network is not arequirement as a non-mesh network will also work. The primary cachedownloads and caches content. The secondary cache synchronizes allcontent from the primary cache and provides failover.

In one embodiment, the roles are determined by a competitive biddingprocess that takes into account several factors including a number ofneighbor nodes; throughput for each node; latency; number of hops;network distance, etc. Bidding can take place among CWSes, or at theLAC. A bidding protocol may be used. SON factors may also beincorporated in the bidding. For example, nodes that serve primarilyvideo UEs may be weighted with a high score, nodes that are unreliablemay be weighted with a low score. Bidding could also be used for anyother role, such as: neighbor relations table handling; selection oflocal EPC; selection of X2 concentrator.

Other aspects and advantages of the invention will become apparent fromthe following drawings, detailed description, and claims, all of whichillustrate the principles of the invention, by way of example only. Afirst mesh network node is designated as a caching node to cache contentfor the UE attached to the first base station. The UE moves from thefirst base station to the second base station. A determination is madewhether the first mesh network node should continue to act as a cachingnode for the UE after it has moved to the second base station and thefirst mesh network node is used as the caching node when thedetermination is that the first mesh network should continue to act asthe caching node for the UE. The second mesh network node is designatedas the caching node to cache content for the UE attached to the secondbase station when the determination is that the first mesh networkshould not continue to act as the caching node for the UE.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the invention and many attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings. In thedrawings, like reference characters generally refer to the same partsthroughout the different views. Further, the drawings are notnecessarily to scale, with emphasis instead generally being placed uponillustrating the principles of the invention.

FIG. 1 depicts a prior art diagram of a UniRAN deployment.

FIG. 2 depicts a prior art diagram of a master cache in a core network.

FIG. 3 depicts a prior art diagram of a split cache in a core network.

FIG. 4 depicts a diagram of a split cache in a core network, inaccordance with some embodiments.

FIG. 5 depicts a flowchart of a method for providing role assignment forcaching, in accordance with some embodiments.

FIG. 6 depicts a diagram of an overlay cache network, in accordance withsome embodiments.

FIG. 7 depicts a flowchart of a second method for providing roleassignment for caching, in accordance with some embodiments.

FIG. 8 is a diagram showing communication flow, in accordance with someembodiments.

FIG. 9 is a diagram of a networking showing breakout in accordance withsome embodiments.

FIG. 10 is a diagram of an eNodeB, in accordance with some embodiments.

FIG. 11 is a diagram of a coordination server, in accordance with someembodiments.

DETAILED DESCRIPTION

Caching data various types of data (for example video data) is usefulfor reducing latency. A video cache for the UE is generally implementedeither as a master cache in the core network or as a split cache in thecore network and radio access network (RAN). A video cache can aid inreducing latency as compared to the scenario where there is no cachehowever, this there may still be latency issues. For example, videocaching at any particular node may be limited by insufficient number ofintra-node links or insufficient bandwidth. Also, when nodes enter andexit, there may be a need to reallocate the caching node.

While the following description is explained with respect to videocaching, it should be understood that caching of other types of datacould also be implemented. These other types of data may include audioservices, phone services, emergency services (rescue, fire, naturaldisaster etc.). safety-critical applications such as remote surgerywhere availability issues can be hazardous, network operations supportsystems for customers' business critical needs and online games wherereal-time lag can be a factor. It is further noted that the reduction oflatency may be a critical component in 5G networks.

The presently described method, apparatus and software for cachingcontent at mesh network nodes includes assigning a first network node asa primary slave caching node to cache content for a user equipment (UE)attached to a first base station. A determination is made that the UEhas moved from the first base station to a second base station, and adetermination is made whether the first network node should continue toact as the primary slave caching node for the UE. When the determinationis made that the first network node should not continue to act as theprimary slave caching node for the UE, a second network node isdesignated as the primary caching node to cache content for the UEattached to a second base station. The determination may be based onnumber of nodes in the network, throughput available at each node,number of users serviced by each node, latency, number of hops, networkdistance, physical storage capacity, etc.

FIG. 1 shows a typical deployment environment 100 for a UniRAN in anoperator network, as is known in the art. The environment includes acollection of UEs 102, 104, 106, 108, 110 and 112. Each UE is connectedto a respective base station 114, 116, 118, 120, 122, and 124. Alsoshown is a Uni Cloud 126 connecting from base stations to a core network128. When the UniRAN functions as a gateway node, it has access to aninter-connect through which it establishes connectivity to the cloud.The interconnect may be wired or wireless (e.g. LTE). When the UniRANfunctions as a mesh node, the UniRAN has no direct connection to thecore or infrastructure provider network. The only means of connecting tothe network is over mesh-links.

FIG. 2 shows a typical deployment environment 200 showing a master cachein a core network, as is known in the art. The environment 200 includesa UE 202 in communication with a base station 204. An EPC 206 is shownin communication with the base statin 204, a cache 208 and the Internet210. In this example, the video cache for the UE is implemented as amaster cache in the core network. The master cache is where unencrypteddata from the PGW is visible or a DPI technique may be used, whichreduces latency as compared to the scenario where there is no cache orwhere the cache is maintained at the content hosting site/server. Thisscenario suffers from latency related issues.

FIG. 3 shows a typical deployment environment 300 showing a split cache,as is known in the art. The environment 300 includes a UE 302 incommunication with a base station 304 which is in communication with aslave cache 306 which is part of a split cache. An EPC 308 is shown incommunication with the base station 304, a master cache 310 also part ofthe split cache, and the Internet 312. The use of a split cacheaddresses the latency issues experienced when using a single level cachemaintained in the core (FIG. 2). This scenario can also work in case ofa network where the eNodeB moves with the UE as in some vehicularimplementations. However, this mode is limited to a macro eNodeB basednetwork where handovers are extremely few as in the case of handover,the entire cache needs to be purged. This does not work well for a smallcell network.

This presently described method and system for caching solves this issuefor a small cell network organized in a mesh topology. A coordinationserver, also referred as LTE Access Controller (LAC), behaves as a meshmanager/controller for a small cell network. The coordination servercontrols a mesh network where routing throughout the small cell networkis optimized periodically. With this optimization the mesh does notbehave like a pure ad-hoc network but an intelligent mesh whereperformance and characteristics approach a managed static network. Thecoordination server behaves as or controls the primary cache in the corenetwork.

Referring to FIG. 4, a deployment environment 400 is shown. Theenvironment 400 includes a UE 402 in communication with a base station404 which is ii communication with a base station 406. Base station 406is in communication with a slave cache 408. A coordination server 410 isshown in communication with base station 406 and master cache 412. AnEPC core network 414 is in communication with the coordination server410 and with the Internet 416.

As the coordination server also manages the mesh, the point at which itdoes the periodic optimization of routing, it is also in a position todesignate the primary slave-cache at a node in the mesh based oncumulative traffic flows for a small-cell. At this point thecoordination server may also designate a secondary slave-cache at analternate node. If required, the coordination server may also introducea tertiary cache in the network at an intermediate mesh-node for alarger mesh.

The coordination server can decide to change the designated cachingnodes at a rate slower than the routing optimization rate. It could dothis to save on the traffic flow generated while moving the cache. Oneof the primary criteria used in such a case would be elasticity in thelatency for the flows. The lowest bound would be determined by thelatency for static UEs where the cache is maintained at the servingeNodeB, and the high-bound would be determined by the latency expectedwhere there is no slave cache.

The process of moving the cache within the mesh should never be worsethan the scenario described earlier. Thus, moving the cache within themesh offers flexibility required in the mesh network of base stations.The proposed solution provides benefit of selecting the mesh networknode having sufficient number of intra-mesh links and/or sufficientbandwidth for caching the content to be streamed to the user equipments.Additionally, since the coordination server manages the mesh networknodes, the coordinating server is up-to-date about the base station thatwas caching the content and leaving the mesh network. The coordinationserver, therefore, may move the cache content to another base station inthe mesh network and update the IP addresses for UE and other processesfor the most recent base station used as a content cache server.

As shown in FIG. 4, the coordination server or the LAC decides todesignate the Gateway node-1 as the primary slave-cache for all the UEsattached to node-2. As the gateway node-1 is the anchoring node formultiple such nodes, when the UE moves from node-2 to node-3 (not shownin the FIG. 4) which is also anchored at node-2, no cache changeshappen. Thus, a UE in general is able to roam over a larger area than atypical small-cell, with similar caching benefits as a macro-cellnetwork.

A flow diagram of a particular embodiment is shown in FIG. 5. Theprocess 500 begins at step 501, assign a gateway mesh node node-1 as aprimary slave-cache server to stream the cached content to the attacheduser equipment (UE) at a base station e.g. node-2. At step 502, the UEmoves and attached another base station e.g. node-3. At step 503, if thebase station (node-3) now serving the UE is also anchored at gatewaynode-1, no change is required for cache content, as shown at step 504.However, if the user equipment is being served by a base station node-3not anchored at node-1, the content needs to be cached at the differentanchoring node, as shown at step 505.

FIG. 6 shows an overlay cache network 600 where the slave-cache is pereNodeB rather than a (group of) UE(s). A UE 602 is in communication witha first eNodeB of a mesh of eNodeBs. The mesh of eNodeBs includeseNodeBs 604, 606, 608, 610, 612 and 614. The mesh is in communicationwith coordinating server 616. An EPC/Core network 620 is incommunication with coordinating server 616 and Internet 622. A splitcache is shown as master cache 626 and slave cache 628. Also shown isslave cache 630 which is implemented for a UE move from RaN-2 to Ran-5in this example.

In an LTE network the UE traffic is tunneled so it is not easy to movetraffic to move caches around using a single layer mesh. Thus, here weoverlay another network on the mesh specifically for cache use. The IPsof this overlay network are moved around the mesh as needed, whilemaintaining the binding to the UEs.

In this overlay network, an IP address is associated with each primaryand secondary slave-cache in the network. Each such pair could serveeither a single UE or a group of UEs with similar mobility and locationcharacteristics. The coordination server or the LAC would maintain atable to bind UE and the cache-IP. An example of the table to bind UEand the cache mesh node's IP address is shown below.

TABLE 1 Mapping Table for UE to Cache Mesh Node IP Address Cache CacheCache Mesh Mesh Mesh Node Node Node Active Standby Virtual UE IP IP IPIdentifier Address Address Address 083901136446086127 10.0.0.1 10.0.0.210.0.0.3

The UE Identifier could be any identifier such as MEID, IMEI, IMSI, etc.The example above uses IMSI as the UE Identifier.

When the LAC determines that it needs to move the slave-caches in thenetwork it just moves the IP address for the secondary-cache to the newmesh node and moves the cache to it. It could also allocate a new serveraddress to which it would force the former secondary to sync to beforeupdating the new secondary location at the primary. At this point, ifdesired a virtual switchover between the primary and secondary slavescould be performed. A mapping table at the LAC would allow for a UEcache to move to a different server if latency thresholds are hit.

As shown in the Table 1 above, there may exist a primary andsecondary/slave base station providing the content to the userequipments. While, the primary base station performs the main role ofdownloading and storing the content for streaming, the secondary basestation may synchronize itself with the primary and may assume the roleof primary base station for content streaming and storing in the casewhen primary base station fails. Any standard protocol forsynchronization may be used between primary and secondary base stationfor content synchronization.

There may be a heartbeat mechanism between the primary and the secondarybase station to detect failure and decide when to perform role switch.Additionally, the decision of which base station may be primary basestation may be either hardcoded or may be decided based on start-upsequence and negotiation between the primary and the secondary basestation. In alternate, the coordination server may act as an arbitratorand decide the primary and secondary base station. The coordinationserver or the base stations may decide primary base station based onnumber of nodes in the mesh network, throughput available at each node,number of users serviced by each node, latency, number of hops, networkdistance, physical storage capacity, etc. The primary and secondary basestations, or the coordination server, may use bidding protocol in roledetermination. While bidding protocol may be used for primary roledetermination for content caching and streaming, the bidding protocolmay be used for role determination for other functions such as neighborrelations table handling, selection of local EPC, selection of localconcentrator, etc.

The base stations and the coordination server may also use the featuresof self-organizing network module for dynamic feature update, role swap,etc. For example, the nodes that are primarily serving video userequipments may be weighted with a higher score in role determination.Similarly, if the node is unreliable or with high mobility andfrequently changing location or joining and leaving the mesh network athigher rate may be given lower score in deciding primary role. In someembodiments, the table may be managed of current active UE transmissionsessions and may be maintained during failover.

As described in FIG. 7, at step 701, a primary and a secondaryslave-cache base station/mesh node for the eNodeB (e.g. eNodeB-1) isassigned at the coordination server. The user equipment attached toeNodeB will be served by the primary and the secondary slave-cache meshnodes, based on the association of the UE identifier and the primary andthe secondary slave-cache base station/mesh node's IP address, where theassociation is stored in a table as shown in Table 1 above at thecoordination server. If the UE moves and attaches to another eNodeB(e.g. eNodeB-2) as shown at step 702, the Table 1 shown above will beupdated with the IP addresses for the primary and the secondary meshnodes' IP address for the eNodeB-2 as shown at 703. At step 705, if theprimary/active cache server has encountered a problem and is unable toperform the role as an active or primary cache server, as shown at step706, primary and secondary mesh nodes' IP addresses are swapped i.e.secondary cache mesh node becomes active and previously active cachemesh node becomes secondary server. However, at step 702, if UE has notmoved and not attached to another eNodeB, or at step 705, the primaryserver is acting as expected, no change is necessary as shown at step704 and 707.

The following example may be useful to understand how to use thedisclosure made in the present application. For example, UE1 is beingserviced by a base station BS1. The base station BS1 and base stationBS2, base station BS3 and base station BS4 may be nodes in the meshnetwork, where base station BS4 may provide access to the core network.The content to be streamed to the UE1 may be stored at BS2 as a primarycache server and BS3 as a secondary cache server. BS2 and BS3 maynegotiate the primary and secondary role using the bidding protocol, orthe coordination server managing this mesh network of base stations BS1,BS2, BS3, and BS4 may determine primary and secondary cache server basedon various factors described above. If the BS2 fails for any reason orstreaming from BS2 to UE1 connected to BS1 may be having latency relatedissues or other issues affecting quality of streaming, the coordinationserver, or the BS2 and BS3 may renegotiate the primary role causing theIP address of the primary and secondary cache server updated.Additionally, BS2 and BS3 may synchronize the content stored usingsynchronization protocol.

As shown in FIG. 8, UE 801 attaches to an eNodeB eNodeB-1 802 at 806.The eNodeB-1 802 is anchored at a caching server caching-server-1 803 asshown at 807 and an eNodeB eNodeB-2 804 is anchored at a caching servercaching-server-2 805 as shown at 808. As shown at 809, content to bestreamed to the UE 801 is cached at the caching server 803. When the UEsends request for content to the eNodeB 802 as shown at 810, the eNodeB802 forwards the request to the caching server 803 as shown at 811. Thecaching server 803 streams the content or sends the response withcontent to be streamed to the eNodeB 802 as shown at 812. The eNodeB 802streams the content to the UE 801 as shown at 813. As shown at 815, whenUE moves and subsequently attaches to eNodeB 804 as shown at 814, thecoordination server updates the cached content from caching server 803to the caching server 804 for the UE 801. As shown at 816, when the UEsends request for content to eNodeB 804, the eNodeB 804 forwards therequest for content to the caching server 805, as shown at 817. Uponreceipt of response from the caching server 805 at the eNodeB 804 asshown at 818, the eNodeB 804 streams the requested content to the UE 801as shown at 819.

FIG. 9 is a network diagram showing the concept of local breakout. Localbreakout refers to a mechanism wherein IP data can be routed to adestination from the source over the Internet, without going through acellular core network gateway as an intermediate node or nodes. As basestations are often provisioned with a connection to the public Internetfor backhaul purposes, local breakout is often possible for handlinggeneric Internet traffic, as well as voice, RTP, or real-time traffic incertain scenarios. Local breakout can greatly reduce network latency ina wireless network by making many connections more direct, which isespecially important in the implementation of 5G networks.

Base stations 902 and 920 are in communication with an HNG 904. HNG 904communicates with a video service 912 or 914 by way of a 4G core networkconnection 906, a 5G core network connection 908 or a 2G/3G core networkconnection 910, or any other core network or combination thereof. Inthis example base station 902 and base station 920 are bidding againsteach other to act as a primary video caching server as describedelsewhere herein. The determination regarding which base station obtainsthe video caching role may take into account several factors, includingbut not limited to, the amount of latency between the base station andthe video service. In this example, base station 902 has lower latencyto the video service than base station 920, and base station 902 isawarded the primary cache role. A breakout connection 916 is shown fromHNG 904 to video services 912 (YouTube) and 914 (Netflix)—these videoservices are representative and any other video services accessed overthe public Internet could be considered. A breakout connection 918 isalso shown from base station 902 to video service 912 and 914. In thecase of video caching for a video server that is located within acarrier core network, the caching server role may be awarded to the basestation with a better connection to the core and not to the publicInternet; different base stations may provide caching for differentvideo services, in some embodiments.

FIG. 10 is a schematic diagram of an enhanced base station, inaccordance with some embodiments. Enhanced base station 1000 may be aneNodeB for use with LTE, and may include processor 1002, processormemory 1004 in communication with the processor, baseband processor1006, and baseband processor memory 1008 in communication with thebaseband processor. Enhanced eNodeB 1000 may also include first radiotransceiver 1010 and second radio transceiver 1012, internal universalserial bus (USB) port 1016, and subscriber information module card (SIMcard) 1018 coupled to USB port 1014. In some embodiments, the secondradio transceiver 1012 itself may be coupled to USB port 1016, andcommunications from the baseband processor may be passed through USBport 1016.

Processor 1002 may communicate with an upstream gateway viaself-organizing network (SON) coordination module 1031, which mayperform the buffering, flow control, radio congestion identification,X2AP communication, and other functions described herein. The SON modulemay be configured to provide transmit power increase/decreasefunctionality, radio band switching functionality, or communicationswith another remote SON module providing, for example, these types offunctionality, in some embodiments.

Baseband processor 1006 may generate and receive radio signals for bothradio transceivers 1010 and 1012, based on instructions from processor1002. In some embodiments, processors 1002 and 1006 may be on the samephysical logic board. In other embodiments, they may be on separatelogic boards.

The first radio transceiver 1010 may be a radio transceiver capable ofproviding LTE eNodeB functionality, and may be capable of higher powerand multi-channel OFDMA. The second radio transceiver 1012 may be aradio transceiver capable of providing LTE UE functionality. Bothtransceivers 1010 and 1012 are capable of receiving and transmitting onone or more LTE bands. In some embodiments, either or both oftransceivers 1010 and 1012 may be capable of providing both LTE eNodeBand LTE UE functionality. Transceiver 1010 may be coupled to processor1002 via a Peripheral Component Interconnect-Express (PCI-E) bus, and/orvia a daughtercard. As transceiver 1012 is for providing LTE UEfunctionality, in effect emulating a user equipment, it may be connectedvia the same or different PCI-E bus, or by a USB bus, and may also becoupled to SIM card 1018.

SIM card 1018 may provide information required for authenticating thesimulated UE to the evolved packet core (EPC). When no access to anoperator EPC is available, local EPC 1020 may be used, or another localEPC on the network may be used. This information may be stored withinthe SIM card, and may include one or more of an international mobileequipment identity (IMEI), international mobile subscriber identity(IMSI), or other parameter needed to identify a UE. Special parametersmay also be stored in the SIM card or provided by the processor duringprocessing to identify to a target eNodeB that device 1000 is not anordinary UE but instead is a special UE for providing backhaul to device1000.

Wired backhaul or wireless backhaul may be used. Wired backhaul may bean Ethernet-based backhaul (including Gigabit Ethernet), or afiber-optic backhaul connection, or a cable-based backhaul connection,in some embodiments. Additionally, wireless backhaul may be provided inaddition to wireless transceivers 1010 and 1012, which may be Wi-Fi1002.11a/b/g/n/ac/ad/ah, Bluetooth, ZigBee, microwave (includingline-of-sight microwave), or another wireless backhaul connection. Anyof the wired and wireless connections may be used for either access orbackhaul, according to identified network conditions and needs, and maybe under the control of processor 1002 for reconfiguration.

Other elements and/or modules may also be included, such as a homeeNodeB, a local gateway (LGW), etc. Additional radio amplifiers, radiotransceivers and/or wired network connections may also be included.

Processor 1002 and baseband processor 1006 are in communication with oneanother. Processor 1002 may perform routing functions in conjunctionwith backhaul/routing coordination module 1030, and may determineif/when a switch in network configuration is needed. Routes, backhaulinformation, and other information may be stored in module 1030 or in adatabase, table, or in-memory data structure in memory 1003.

Processor 1002 may identify the appropriate network configuration, andmay perform routing of packets from one network interface to anotheraccordingly. Processor 1002 may use memory 1004, in particular to storea routing table to be used for routing packets, and may use memory 1004to store one or more routing table of routing coordination module 1030.Routes may be pushed to other mesh nodes as required by rules inbackhaul/routing coordination module 1030 or based on instructionsreceived at SON coordination module 1031 from other nodes or from anupstream coordinating server.

Baseband processor 1006 may perform operations to generate the radiofrequency signals for transmission or retransmission by bothtransceivers 1010 and 1012. Baseband processor 1006 may also performoperations to decode signals received by transceivers 1010 and 1012.Baseband processor 1006 may use memory 1008 to perform these tasks.

FIG. 11 is a schematic diagram of a coordination gateway, in accordancewith some embodiments. Coordinating gateway 1100 includes processor 1102and memory 1104, which are configured to provide the functions describedherein. Also present are radio access network coordination/X2/X2+signaling (RAN Coordination and signaling) module 1106, back to backuser agent (B2BUA) and media buffer 1108, and core networkvirtualization module 1110. In some embodiments, coordinator server 1100may coordinate multiple RANs using coordination module 1106. In someembodiments, coordination server may also provide proxying, routingvirtualization and RAN virtualization, via module 1110. In someembodiments, buffering and proxying of media may be performed by module1108. In some embodiments, a downstream network interface 1112 isprovided for interfacing with the RANs, which may be a radio interface(e.g., LTE), and an upstream network interface 1114 is provided forinterfacing with the core network, which may be either a radio interface(e.g., LTE) or a wired interface (e.g., Ethernet). Signaling suppressionfunctions as described within may be performed in module 1106.

Coordinating gateway 1100 includes local evolved packet core (EPC)module 1120, for authenticating users, storing and caching priorityprofile information, and performing other EPC-dependent functions whenno backhaul link is available. Local EPC 1120 may include local HSS1122, local MME 1124, local SGW 1126, and local PGW 1128, as well asother modules. Local EPC 1120 may incorporate these modules as softwaremodules, processes, or containers. Local EPC 1120 may alternativelyincorporate these modules as a small number of monolithic softwareprocesses. Modules 1106, 1108, 1110 and local EPC 1120 may each run onprocessor 1102 or on another processor, or may be located within anotherdevice.

The foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. As will be understood by thoseskilled in the art, the present invention may be embodied in otherspecific forms without departing from the spirit or essentialcharacteristics thereof. Various components in the devices describedherein may be added, removed, or substituted with those having the sameor similar functionality. Various steps as described in the figures andspecification may be added or removed from the processes describedherein, and the steps described may be performed in an alternativeorder, consistent with the spirit of the invention. Accordingly, thedisclosure of the present invention is intended to be illustrative, butnot limiting of the scope of the invention, as well as other claims. Thedisclosure, including any readily discernible variants of the teachingsherein, defines, in part, the scope of the foregoing claim terminology.

While the examples are described with respect to a mesh network, itshould be understood that the same concepts apply to a non-mesh network.Where the word “mesh” is used herein, it is understood to mean anycollection of nodes that have dynamic routing characteristics, includingstatically-planned nodes with wireless backhaul (such backhaul linkshave rapidly changing characteristics) as well as statically plannednodes with static backhaul where nodes are anticipated to become morecongested, causing them to become more or less available; in otherwords, the video caching role assignment procedure described herein isapplicable to any general selection of nodes in a cellular communicationnetwork. The RF tower may be statically or dynamically planned, howeverin either case the video caching roles can be self-configuring. Thesystem works in any RAT or multi-RAT environment, including but notlimited to 5G with video caching role assignment taking place at theedge of the network.

It is understood that any specific order or hierarchy of steps in theprocesses disclosed is an illustration of example approaches. Based upondesign preferences, it is understood that the specific order orhierarchy of steps in the processes may be rearranged, or that allillustrated steps be performed. Some of the steps may be performedsimultaneously. For example, in certain circumstances, multitasking andparallel processing may be advantageous. Moreover, the separation ofvarious system components illustrated above should not be understood asrequiring such separation, and it should be understood that thedescribed program components and system can generally be integratedtogether in a single software product or packaged into multiple softwareproducts.

The above-described features and applications can be implemented assoftware processes that are specified as a set of instructions recordedon a computer-readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or moreprocessing unit(s) (e.g. one or more processors, cores of processors, orother processing units), they cause the processing unit(s) to performthe actions indicated in the instructions. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, harddrives, RAM chips, EPROMs, etc. The computer-readable media does notinclude carrier waves and electronic signals passing wirelessly or wiredconnections. Code may be written in any combination of programminglanguages or machine-readable data formats, each suitable to itsparticular application, including but not limited to: C, C++, Java,Python, Ruby, R, Lua, Lisp, Scala, JSON, JavaScript, YAML, XML, HTML,etc. Services may be RESTful and may be implemented using generic hooks,including over HTTP, HTTPS, SCTP, IP, TCP, JSON, JavaScript, etc., aswell as via inter-process communication on one or more real or virtualmachines or containers, e.g., IPC, shared memory, shared filesystem,UNIX pipes and the like. A Linux or POSIX environment may be used.Containers may be Docker, Jetty, Tomcat, Wildfy, Springboot, LXD,unikernels, OpenVZ, RKT, Windows Server, Hyper-V, or any other type ofcontainer, or may be, in some embodiments, virtual machines or images,etc. Network access may be relied upon or may be avoided, in variousembodiments. A networking fabric may be provided among the differentcontainers, in some embodiments. As is well-known, the benefit of usingcloud infrastructure is that it is simple to mix heterogeneous resourcesand to scale services up or down based on load and desired performance.

In the specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storageor flash storage, for example, a solid-state drive, which can be readinto memory for processing by a processor. Also, in someimplementations, multiple software technologies can be implemented assub-parts of a larger program while remaining distinct softwaretechnologies. In some implementations, multiple software technologiescan also be implemented as separate programs. Finally, any combinationof separate programs that together implement a software technologydescribed here is within the scope of the subject technology. In someimplementations, the software programs, when installed to operate on oneor more electronics systems, define one or more specific machineimplementations that execute and perform the operations of the softwareprograms.

A computer program (also known as program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astandalone program or as a module, component, subroutine, object, oranother unit suitable for use in a computing environment. A computerprogram may, but need not correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

These functions described above can be implemented in digital electroniccircuitry, in computer software, hardware, or firmware. The techniquescan be implemented using one or more computer program products.Programmable processors and computers can be included in or packaged asmobile devices. The process and logic flows can be performed by one ormore programmable processors and by one or more programmable logiccircuitry. General and special purpose computing devices and storagedevices can be interconnected through communication networks.

Some implementations include electronic components, for examplemicroprocessors, storage and memory that store computer programinstructions in a machine-readable or computer-readable medium(alternatively referred to as computer-readable storage media,machine-readable media, or machine-readable storage media). Someexamples of such computer-readable media include RAM, ROM, read-onlycompact discs (CD-ROM), readable compact discs (CD-R), rewritablecompact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM,dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g.DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SDcards, micro-SD cards, etc.), magnetic or solid-state hard drives,read-only and recordable Blu-Ray® discs, ultra-density optical discs,any other optical or magnetic media, and floppy disks. Thecomputer-readable media can store a computer program that is executed byat least one processing unit and includes sets of instructions forperforming various operations. Examples of computer programs or computercode include machine code, for example is produced by a compiler, andfiles including higher-level code that are executed by a computer, anelectronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some implementations areperformed by one or more integrated circuits, for example applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some implementations, such integrated circuits executeinstructions that are stored in the circuit itself.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purpose of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer-readable media” and “computer readable medium” are entirelyrestricted to tangible, physical objects that store information in aform that is readable by a computer. These terms exclude any wireless,wired download signals, and any other ephemeral signals.

To provide for interaction with a user, implementations of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, or any other available monitor types, fordisplaying information to the user and a keyboard and a pointing device,e.g., mouse or trackball, by which the user can provide input to thecomputer. Other kinds of devices can be used to provide for interactionwith a user as well; for example, feedback provided to the user can beany form of sensory feedback, e.g., visual feedback, tactile feedback,or auditory feedback; and input from the user can be received in anyform, including acoustic, speech, or tactile input. In addition, acomputer can interact with a user by sending documents to and receivingdocuments from a device that is used by the user; for example, bysending web pages to a web browser on a user's client device in responseto requests received from the web browser.

The subject matter described in this specification can be implemented ina computing system that includes a back-end component, e.g., as a dataserver, or that includes a middleware component, e.g., an applicationserver, or that includes a front-end component, e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the subject matter described inthis specification, or any combination of one or more such back-end,middleware, or front-end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication network includea local area network (“LAN”), an inter-network (e.g., the Internet), andpeer-to-peer networks (e.g., ad-hoc peer-to-peer networks).

The subject matter described in this specification can be implementedusing client-side applications, web pages, mobile web pages, or othersoftware as generally known in the art and that would be usable toend-user customers (for community self-managed RAN apps) and/or mobileoperator end users. The subject matter could alternately be delivered orimplemented using an API, such as a SOAP API, a JSON API, a RESTful API,in lieu of or in conjunction with a direct end-user interface. Thesubject matter could use messaging queues, webhooks, server-sidecontainers, or any other technology known in the art.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someaspects of the disclosed subject matter, a server transmits data (e.g.,an HTML page) to a client device (e.g., for purpose of displaying datato and receiving user input from a user interacting with the clientdevice). Data generated at the client device (e.g., a result of the userinteraction) can be received from the client device at the server. Anydatabase could be used (SQL, NoSQL, temporal, key-value, etc.). Anycontainer orchestration technology (Kubernetes, Docker Swarm) could beused.

Various modifications to these aspects will be readily apparent, and thegeneric principles defined herein may be applied to other aspects. Thus,the claims are not intended to be limited to the aspects shown herein,but is to be accorded the full scope consistent with the languageclaims, where reference to an element in singular is not intended tomean “one and only one” unless specifically so states, but rather “oneor more.” Unless expressly stated otherwise, the term “some” refers toone or more. Pronouns in the masculine (e.g., his) include the feminineand neuter gender (e.g., her and its) and vice versa. Headings andsubheadings, if any, are used for convenience only, and do not limit thesubject technology.

A phrase, for example, an “aspect” does not imply that the aspect isessential to the subject technology or that the aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations. Aphrase, for example, an aspect may refer to one or more aspects and viceversa. A phrase, for example, a “configuration” does not imply that suchconfiguration is essential to the subject technology or that suchconfiguration applies to all configurations of the subject technology. Adisclosure relating to a configuration may apply to all configurationsor one or more configurations. A phrase, for example, a configurationmay refer to one or more configurations and vice versa.

The foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. In some embodiments, softwarethat, when executed, causes a device to perform the methods describedherein may be stored on a computer-readable medium such as a computermemory storage device, a hard disk, a flash drive, an optical disc, orthe like. As will be understood by those skilled in the art, the presentinvention may be embodied in other specific forms without departing fromthe spirit or essential characteristics thereof. For example, cloudtopology could vary and public and private cloud services could bemixed; certain services could be provided by containers while otherservices could be provided by dedicated machines or virtual machines orvirtual network functions (for example, a data sink could be atraditional billing server); wireless network topology can also apply towired networks, optical networks, and the like; etc. The methods mayapply to LTE-compatible networks, to UMTS-compatible networks, or tonetworks for additional protocols that utilize radio frequency datatransmission. Various components in the devices described herein may beadded, removed, or substituted with those having the same or similarfunctionality. Various steps as described in the figures andspecification may be added or removed from the processes describedherein, and the steps described may be performed in an alternativeorder, consistent with the spirit of the invention. Accordingly, thedisclosure of the present invention is intended to be illustrative of,but not limiting of, the scope of the invention, which is specified inthe following claims.

What is claimed is:
 1. A method for caching content at network nodes,comprising assigning a first network node as a primary slave cachingnode to cache content for a user equipment (UE) attached to a first basestation; determining, by a coordinating server, the UE has moved fromthe first base station to a second base station; determining, by thecoordinating server, whether the first network node should continue toact as the primary slave caching node for the UE after it has moved tothe second base station; continuing use of the first network node as theprimary slave caching node when the determination is that the firstnetwork node should continue to act as the primary slave caching nodefor the UE; and designating a second network node as the primary slavecaching node to cache content for the UE attached to a second basestation when the determination is that the first network node should notcontinue to act as the primary slave caching node for the UE, whereinthe first and second network nodes are located in a Radio Access Network(RAN).
 2. The method of claim 1, further comprising moving the cachedcontent from first network node to the second network node when thecaching node is the second network node.
 3. The method of claim 1,wherein determining, by a coordinating server, the UE has moved from thefirst base station to a second base station comprises determining thatthe second base station is anchored at a different slave caching node 4.The method of claim 1, further comprising assigning a secondary slavecaching node to cache content for the user equipment (UE)
 5. The methodof claim 4, further comprising associating the UE to the primary slavecaching node and to the secondary slave caching node IP address at thecoordinating server.
 6. The method of claim 5, wherein designating asecond network node as the primary caching node to cache content for theUE attached to a second base station when the determination is that thefirst network node should not continue to act as the primary slavecaching node for the UE further comprises updating the primary and thesecondary nodes IP address for the UE identifier.
 7. The method of claim5, further comprising determining whether the primary slave caching nodehas stopped functioning, when the determination is that the primaryslave caching node has not stopped functioning then refraining fromchanging anything, and when the determination is that the primary slavecaching node has stopped functioning then swapping the primary andsecondary caching nodes IP address for the UE identifier.
 8. Anon-transitory computer-readable medium containing instructions which,when executed, cause a system to perform steps comprising: assigning afirst network node as a primary slave caching node to cache content fora user equipment (UE) attached to a first base station; determining, bya coordinating server, the UE has moved from the first base station to asecond base station; determining, by the coordinating server, whetherthe first network node should continue to act as the primary slavecaching node for the UE after it has moved to the second base station;continuing use of the first network node as the primary slave cachingnode when the determination is that the first network node shouldcontinue to act as the primary slave caching node for the UE; anddesignating a second network node as the primary slave caching node tocache content for the UE attached to a second base station when thedetermination is that the first network node should not continue to actas the primary slave caching node for the UE, wherein the first andsecond network nodes are located in a Radio Access Network (RAN).
 9. Thecomputer-readable medium of claim 8, further comprising instructions formoving the cached content from first network node to the second networknode when the caching node is the second network node.
 10. Thecomputer-readable medium of claim 8, further comprising instructionswherein determining, by a coordinating server, the UE has moved from thefirst base station to a second base station comprises determining thatthe second base station is anchored at a different slave caching node11. The computer-readable medium of claim 8, further comprisinginstructions for assigning a secondary slave caching node to cachecontent for the user equipment (UE)
 12. The computer-readable medium ofclaim 11, further comprising instructions for associating the UE to theprimary slave caching node and to the secondary slave caching node IPaddress at the coordinating server.
 13. The computer-readable medium ofclaim 12, further comprising instructions wherein designating a secondnetwork node as the primary slave caching node to cache content for theUE attached to a second base station when the determination is that thefirst network node should not continue to act as the primary slavecaching node for the UE further comprises updating the primary and thesecondary nodes IP address for the UE identifier.
 14. Thecomputer-readable medium of claim 12, further comprising instructionsfor determining whether the primary slave caching node has stoppedfunctioning, when the determination is that the primary slave cachingnode has not stopped functioning then refraining from changing anything,and when the determination is that the primary slave caching node hasstopped functioning then swapping the primary and secondary cachingnodes IP address for the UE identifier.
 15. A system for caching contentat the network nodes, comprising a first network node assigned as aprimary slave caching node to cache content for a user equipment (UE)attached to a first base station; a coordinating server, for determiningthe UE has moved from the first base station to a second base stationand determining whether the first network node should continue to act asthe primary slave caching node for the UE after it has moved to thesecond base station; the first network node continuing use as theprimary slave caching node when the determination is that the firstnetwork should continue to act as the primary slave caching node for theUE; and a second network node used as the primary caching node to cachecontent for the UE attached to a second base station when thedetermination is that the first network should not continue to act asthe caching node for the UE, wherein the first and second network nodesare located in a Radio Access Network (RAN).
 16. The system of claim 15,wherein the cached content is moved from the first network node to thesecond network node when the caching node is the second network node.17. The system of claim 15, wherein determining the UE has moved fromthe first base station to a second base station comprises determiningthat the second base station is anchored at a different slave cachingnode.
 18. The system of claim 15, wherein a secondary slave cache nodeis assigned to cache content for the user equipment (UE) and wherein theUE is associated to the primary slave caching node and to the secondaryslave caching node IP address at the coordinating server.
 19. The systemof claim 18, wherein a second network node is designated as the primarycaching node to cache content for the UE attached to a second basestation when the determination is that the first network should notcontinue to act as the caching node for the UE further comprisesupdating the primary and the secondary nodes IP address for the UEidentifier.
 20. The system of claim 18, wherein when the determinationis that the primary slave caching node has not stopped functioning thennothing is changed, and when the determination is that the primary slavecaching node has stopped functioning then the primary and secondarycaching nodes IP address is swapped for the UE identifier.